Method of estimating an affine relation between images

ABSTRACT

A method ( 400 ) is disclosed for estimating an affine relation between a first image and a second image. The first and second images each have at least 4 non-parallel lines therein. The method ( 400 ) starts by identifying ( 406 ) sets of intersection points of the lines appearing in each of the images. The method ( 400 ) then determines ( 412 ) whether a relation between intersection points exists. If the relation exists then the first and second images are affine related and the affine distortion may be inverted ( 418 ).

FIELD OF THE INVENTION

The present invention relates generally to image processing and, inparticular, to estimating an affine relation between images.

BACKGROUND

Watermarks are often used to embed information into an imageimperceptibly. However, many of the watermarks in general use aredestroyed by distortions, such as scaling, rotation, shearing, andanamorphic scaling. Some watermark detection methods are even sensitiveto translations.

Various methods have been proposed for detecting watermarks, with someof those methods said to be invariant to affine distortions. However,most methods that are invariant to affine distortions require, at somestage, an extensive search through a space of parameters defining theaffine distortions. Such searches may cover many dimensions andtypically consume considerable computing capacity.

Recently some methods have been proposed that reduce the search space byperforming transformations which first remove the translation, thenconvert scaling and rotation into further translation effects in twoorthogonal directions. These methods are known as RST (Rotation, Scale,and Translation) invariant methods. Typically such techniques requirecomplementarity of the embedding and detection procedures.

Other techniques rely on embedding patterns with special symmetryproperties, such as rotational symmetry, and then detecting thosepatterns by extensive search over the non-symmetric distortionparameter(s).

As none of the aforementioned methods imbue full affine distortioninvariance, an extensive search through one or more of the distortionparameter spaces is still required to ensure full invariance.Accordingly, there is a need for a method of estimating the parametersof the affine distortion without requiring an extensive search trough aparameter space.

SUMMARY

It is an object of the present invention to substantially overcome, orat least ameliorate, one or more disadvantages of existing arrangements.

Disclosed are arrangements which seek to address the above problems. Thearrangements estimate parameters of affine distortions applied to animage by identifying correspondences between intersection points oflines embedded into the image and that of a distorted version thereof.

According to an aspect of the present invention, there is provided amethod for estimating an affine relation between first and secondimages, said images each having at least 4 non-parallel lines therein,said method comprising the steps of:

identifying a first set of intersection points of said lines appearingin said first image;

identifying a second set of intersection points of said lines appearingin said second image;

determining whether a relation between intersection points from saidfirst and second sets of intersection points points exists, wherein ifsaid relation exists then said first and second images are affinerelated.

According to yet another aspect of the present invention, there isprovided a method of detecting an auxiliary pattern in a second image,said second image being an affine distorted version of a first imagehaving said auxiliary pattern and at least 4 non-parallel lines therein,said lines forming a first set of intersection points at predeterminedpositions, said method comprising the steps of:

identifying a second set of intersection points of said lines appearingin said second image;

identifying relation between intersection points from said first andsecond sets of intersection points;

estimating said affine distortion parameters using said first and secondsets of intersection points;

applying said affine distortions to said auxiliary pattern to form adistorted auxiliary pattern, said affine distortions being defined bysaid affine distortion parameters; and

detecting said distorted auxiliary pattern in said second image.

According to yet another aspect of the present invention, there isprovided an apparatus for implementing any one of the aforementionedmethods.

According to another aspect of the present invention there is provided acomputer program product including a computer readable medium havingrecorded thereon a computer program for implementing any one of themethods described above.

Other aspects of the invention are also disclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present invention will now be describedwith reference to the drawings, in which:

FIG. 1A shows a straight-line embedded into an image and the parametersdefining the straight-line;

FIG. 1B illustrates the parameters defining the straight-line of FIG. 1Ain the polar space;

FIGS. 2A and 2B show the parameter pairs of the same 3 lines in thepolar space before distortion and after distortion;

FIG. 3 shows a schematic flow diagram of a method of detectingparameters of an affine distortion from an image;

FIG. 4 shows a schematic flow diagram of a method of embedding thestraight-lines using watermarks;

FIG. 5 is a schematic block diagram of a general purpose computer uponwhich arrangements described can be practiced;

FIG. 6 shows 4 lines arranged such that the orientation angles β_(n) ofthe lines are uniformly distributed over the fall angular range;

FIG. 7 shows the length ratios of line segments as a function of theparameter e;

FIG. 8 illustrates an example pattern that may be used as a watermark;

FIG. 9 illustrates the projection of the undistorted example patternillustrated in FIG. 8, plotted in the quasi-polar space;

FIG. 10 shows a schematic flow diagram of a method of estimating whethera second image is an affine distorted version of a first image.

DETAILED DESCRIPTION INCLUDING BEST MODE

Where reference is made in any one or more of the accompanying drawingsto steps and/or features, which have the same reference numerals, thosesteps and/or features have for the purposes of this description the samefunction(s) or operation(s), unless the contrary intention appears.

Consider a straight-line n in an image, an example of which isillustrated in FIG. 1A. It would be understood by a skilled person inthe art that the straight-line n may be represented in a number of ways.For example, two points may be used to represent the straight-line n.The straight-line n may also be embedded into the image, or associatedwith one or more features in the image.

The straight-line n is at an angle β_(n) with the vertical Cartesianaxis (Y-axis) and a distance r_(n) from the Cartesian origin in adirection perpendicular to the straight-line n. The straight-line n isuniquely described by the parameter pair {r_(n),β_(n)}. FIG. 1Billustrates the parameter pair {r_(n),β_(n)} defining the straight-linen in the polar space.

Referring again to FIG. 1A, a point (x,y) on the straight-line it and adistance λ_(n) along the straight-line n may be written in canonicalform as: $\begin{matrix}\left\{ \begin{matrix}{x = {{r_{n}\cos\quad\beta_{n}} - {\lambda_{n}\sin\quad\beta_{n}}}} \\{y = {{r_{n}\sin\quad\beta_{n}} + {\lambda_{n}\cos\quad\beta_{n}}}}\end{matrix} \right. & (1)\end{matrix}$

Now consider a general affine distortion applied to the image includingthe straight-line n. The affine distortion may include one or more ofrotation, scaling, shearing, reflection, translation, and anamorphicscaling. Let ({tilde over (x)},{tilde over (y)}) be the transformedcoordinates in the image of the original point (x,y) after the affinedistortion, then point ({tilde over (x)},{tilde over (y)}) may bewritten as: $\begin{matrix}{\begin{pmatrix}\overset{\sim}{x} \\\overset{\sim}{y}\end{pmatrix} = {{\begin{pmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{pmatrix}\begin{pmatrix}x \\y\end{pmatrix}} + {\begin{pmatrix}x_{0} \\y_{0}\end{pmatrix}.}}} & (2)\end{matrix}$

wherein a_(ij) are parameters defining rotation, scaling, shearing,reflection, and anamorphic scaling, and (x₀,y₀) defines a translation.

It is assumed that ${\det\begin{pmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{pmatrix}} \neq 0.$Furthermore, it is noted that ${\det\begin{pmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{pmatrix}} < 0$indicates that a reflection has occurred.

The total distortion applied by the parameters a_(ij) may be decomposedinto convenient combinations of the prime distortions, namely rotation,anamorphic scaling and shear as follows: $\begin{matrix}{\begin{pmatrix}a_{11} & a_{12} \\a_{21} & a_{22}\end{pmatrix} = {\begin{pmatrix}A & 0 \\0 & B\end{pmatrix}\begin{pmatrix}1 & R \\0 & 1\end{pmatrix}{\begin{pmatrix}{\cos\quad\omega} & {\sin\quad\omega} \\{{- \sin}\quad\omega} & {\cos\quad\omega}\end{pmatrix}.\quad{Transpose}}}} & (3)\end{matrix}$wherein rotation by an angle ω is given by matrix: $\begin{matrix}\begin{pmatrix}{\cos\quad\omega} & {\sin\quad\omega} \\{{- \sin}\quad\omega} & {\cos\quad\omega}\end{pmatrix} & (4)\end{matrix}$anamorphic scaling along the X— and Y-axes is sometimes called aspectratio change and has the form: $\begin{matrix}\begin{pmatrix}A & 0 \\0 & B\end{pmatrix} & (5)\end{matrix}$shear in the x direction has the form: $\begin{matrix}\begin{pmatrix}1 & R \\0 & 1\end{pmatrix} & (6)\end{matrix}$and the transposition, which is applied to place the rotation distortionin the correct quadrant so that |ω|≦45°, is selected from one of thefour options: $\begin{matrix}{{{Transpose} = \begin{pmatrix}0 & 1 \\{- 1} & 0\end{pmatrix}},\begin{pmatrix}{- 1} & 0 \\0 & {- 1}\end{pmatrix},\begin{pmatrix}0 & {- 1} \\1 & 0\end{pmatrix},{\begin{pmatrix}1 & 0 \\0 & 1\end{pmatrix}.}} & (7)\end{matrix}$

Affine distortions have the property that any straight line is preservedduring affine distortion, along with the parallelism of lines, whilelengths and angles change. Accordingly, the straight-line n in the imageis transformed to a straight-line in the deformed image, but thestraight-line in the deformed image is defined by parameter pair {{tildeover (r)}_(n),{tilde over (β)}_(n)}, which is typically different fromthe parameter pair {r_(n),β_(n)} of the straight-line n before theaffine distortions.

From Equations (1) and (2), and using the abbreviations c_(n)=cosβ_(n),s_(n)=sin β_(n), the following relations exist: $\begin{matrix}{{{\tan\quad{\overset{\sim}{\beta}}_{n}} = \frac{\left( {{a_{11}s_{n}} - {a_{12}c_{n}}} \right)}{\left( {{{- a_{21}}s_{n}} + {a_{22}c_{n}}} \right)}}{and}} & (8) \\{{\overset{\sim}{r}}_{n} = {{r_{n}\frac{\begin{matrix}{{\left( {{a_{11}c_{n}} + {a_{12}s_{n}}} \right)\left( {{{- a_{21}}s_{n}} + {a_{22}c_{n}}} \right)} -} \\{\left( {{a_{21}c_{n}} + {a_{22}s_{n}}} \right)\left( {{{- a_{11}}s_{n}} + {a_{12}c_{n}}} \right)}\end{matrix}}{\sqrt{\left( {{a_{11}c_{n}} + {a_{12}s_{n}}} \right)^{2} + \left( {{a_{11}c_{n}} + {a_{12}s_{n}}} \right)^{2}}}} + \frac{{x_{0}\left( {{{- a_{21}}s_{n}} + {a_{22}c_{n}}} \right)} + {y_{0}\left( {{a_{11}c_{n}} + {a_{12}s_{n}}} \right)}}{\sqrt{\left( {{a_{11}c_{n}} + {a_{12}s_{n}}} \right)^{2} + \left( {{a_{11}c_{n}} + {a_{12}s_{n}}} \right)^{2}}}}} & (9)\end{matrix}$

With Equations (8) and (9) including 6 unknown parameters {α₁₁, α₁₂,α₂₁, α₂₂, x₀, y₀} defining the affine distortion, by having 3straight-lines in the image, with each straight line n being defined bythe parameter pair {r_(n)/β_(n)} by detecting the 3 straight-lines inthe affine distorted image, by determining the parameter pair {{tildeover (r)}_(n),{tilde over (β)}_(n)} for each affine distorted line, andby solving Equations (8) and (9), the parameters {α₁₁, α₁₂, α₂₁, α₂₂,x₀, y₀} may be calculated. In order to distinguish reflection, aleft-right asymmetry is also required when placing the straight-linesinto the image.

However, from Equations (8) and (9) it can be seen that, when an affinedistortion is applied to a straight-line, then the parameter pair{{tilde over (r)}_(n),{tilde over (β)}_(n)} is generally transformed ina rather complicated and ambiguous manner. The ambiguity occurs becausea translation or scaling in the direction parallel to the straight-linen has no effect on the parameter pair {r_(n),β_(n)}.

Also, identifying the distorted straight-line corresponding to thestraight-line n in the image before affine distortion is non-obvious. Toillustrate this, FIG. 2A shows the parameter pairs {r_(n),β_(n)} of 3lines in the polar space before distortion, while FIG. 2B shows theparameter pairs {{tilde over (r)}_(n),{tilde over (β)}_(n)} of the same3 lines after distortion. The correspondence between parameter pairs{r_(n),β_(n)} and {{tilde over (r)}_(n),{tilde over (β)}_(n)} before andafter distortion is potentially ambiguous, leading to degeneracy andpossible uncertainty.

A number of techniques may be used to uniquely identify thecorrespondence between each straight-line before and after affinedistortion. One technique is to embed lines where the lines themselvesare unique, and where the uniqueness of the lines are not destroyed byaffine distortions. For example, lines having different styles, colors,etc. may be used to uniquely identify each straight-line.

Another useful property of affine distortions is that the ratio of twosegments of a straight-line is invariant with respect to affinedistortions. In the preferred implementation it is this property that isused to uniquely identify the correspondence between each straight-linebefore and after affine distortion.

Consider N straight-lines in an image, each of the straight-lines againbeing uniquely described by the parameter pair {r_(n),β_(n)}, n=1→N,with angles β_(n)≠β_(m) when n≠m. Hence the straight-lines arenon-parallel.

Because the straight-lines are non-parallel, each straight-lineintersects each other straight-line along its length. The maximum numberof intersection points for N (non-parallel) straight-lines is thetriangular number N(N−1)/2. The point (x_(km),y_(km)) wherestraight-line m intersects straight-line k is: $\begin{matrix}\left\{ \begin{matrix}{x_{km} = {x_{m} = x_{k}}} \\{y_{km} = {y_{m} = y_{k}}}\end{matrix}\Rightarrow\left\{ {\begin{matrix}{{{r_{m}\cos\quad\beta_{m}} - {\lambda_{mk}\sin\quad\beta_{m}}} = {{r_{k}\cos\quad\beta_{k}} - {\lambda_{km}\sin\quad\beta_{k}}}} \\{{{r_{m}\sin\quad\beta_{m}} + {\lambda_{mk}\cos\quad\beta_{m}}} = {{r_{k}\sin\quad\beta_{k}} + {\lambda_{km}\cos\quad\beta_{k}}}}\end{matrix}.} \right. \right. & (10)\end{matrix}$

wherein λ_(km) is the distance along straight-line k where straight-linem intersects. Similarly, distance λ_(km) is the distance alongstraight-line m where straight-line k intersects.

Solving the simultaneous equations in Equation (10), the distance λ_(km)is given by: $\begin{matrix}{\lambda_{km} = {\frac{{r_{k}{\cos\left( {\beta_{k} - \beta_{m}} \right)}} - r_{m}}{\sin\left( {\beta_{k} - \beta_{m}} \right)}.}} & (11)\end{matrix}$

Consider the case where the number of straight-lines N=4. Each line kwould have 3 intersection points (x_(km),y_(km)) with the other 3 linesin. At this stage it is useful to order the distances λ_(km) alongstraight-line k to the respective intersection points (x_(km),y_(km)) bysize as follows:{λ_(km)}_(max)>{λ_(km)}_(mid)>{λ_(km)}_(min) , m=1→4, m≠k   (12)

With segment lengths ξ_(k1) and ξ_(k2) being defined as follows:ξ_(k1)={λ_(km)}_(max)−{λ_(km)}_(mid); andξ_(k2)={λ_(km)}_(mid)−{λ_(km)}_(min)   (14)

a length ratio R_(k) for line k is then defined as: $\begin{matrix}{R_{k} = {{\min\left\lbrack {\frac{\xi_{k\quad 1}}{\xi_{k\quad 2}},\frac{\xi_{k\quad 2}}{\xi_{k\quad 1}}} \right\rbrack} \leq 1}} & (15)\end{matrix}$

With the parameter pairs {r_(n),β_(n)} of the straight-lines suitablychosen, the ratio R_(k) of each line k is distinct from that of everyother line m. Because the ratio R_(k) is invariant with respect toaffine distortions, the correspondence between each straight-line beforeand after affine distortion can be uniquely determined.

Once the corresponding straight-lines have been identified, then thedistances λ_(km) along different straight-lines may be compared beforeand after affine distortion to estimate the scaling factors along eachdirection represented by the straight-line k. Also, the change inorientation of each straight-line may be used for estimation of arotation component of the affine distortion.

Consider a particular case of a rotation followed by an anamorphicscaling applied to the original image, and as is defined in Equations(4) and (5) respectively. The anamorphic scaling has a scaling of factorA in a direction parallel to the horizontal Cartesian axis (X-axis), anda scaling of factor B in a direction perpendicular thereto, with theangle ω being the angle of rotation. In general, a line segment oflength l_(n) at an angle β_(n) will be mapped to a line segment oflength {tilde over (l)}_(n) at an angle {tilde over (β)}_(n), where:$\begin{matrix}{\left\lbrack {\overset{\sim}{l}}_{n} \right\rbrack^{2^{\prime}} = {\left\lbrack l_{n} \right\rbrack^{2}\left\{ {\left\lbrack {A\quad{\cos\left( {\beta_{n} + \omega} \right)}} \right\rbrack^{2} + \left\lbrack {B\quad{\sin\left( {\beta_{n} + \omega} \right)}} \right\rbrack^{2}} \right\}}} & (16) \\{{\overset{\sim}{\beta}}_{n} = {{{arc}\quad\tan\frac{A\quad{\sin\left( {\beta_{n} + \omega} \right)}}{B\quad{\cos\left( {\beta_{n} + \omega} \right)}}} - \omega}} & (17)\end{matrix}$

Equations (16) and (17) contain 3 unknowns, namely scaling factors A andB, and angle ω. Having 4 lines embedded in the original image, and usingthe distance between the outer intersection points of each line as thelength l_(n) as follows:l _(k)={λ_(km)}_(max)−{λ_(km)}_(min) , k=1→4   (18)

4 pairs of equations are obtained. Such an over-constrained system canbe advantageously solved using a least squares method.

It is noted that Equations (16) and (17) are translation invariant, anddo not make provision for translation (x₀,y₀). Also, for all theparameters of the general affine distortion defined in Equation (2) tobe calculated, the matrices of Equations (4) and (5) combined, as inEquation (3), but with the shear factor R being 0.

Alternatively, from the correspondence between each straight-line beforeand after affine distortion, the correspondence between the parameterpairs {r_(n),β_(n)} and {{tilde over (r)}_(n),{tilde over (β)}_(n)} isuniquely determined. Substituting Equation (11) into Equation (10), theintersection point (x_(km),y_(km)) is given by: $\begin{matrix}{\begin{pmatrix}x_{km} \\y_{km}\end{pmatrix} = {\frac{1}{\sin\left( {\beta_{k} - \beta_{m}} \right)}\begin{pmatrix}{\sin\quad\beta_{k}} & {{- \sin}\quad\beta_{m}} \\{\cos\quad\beta_{k}} & {\cos\quad\beta_{m}}\end{pmatrix}{\begin{pmatrix}r_{m} \\r_{k}\end{pmatrix}.}}} & (19)\end{matrix}$

Because the straight-lines are non-parallel, sin(β_(k)−β_(m))≠0. Inpractical situations respective angles β_(k) and β_(m) are chosen suchthat sin²({tilde over (β)}_(k)−{tilde over (β)}_(m))≧0.25 holds.Equivalently, the angle at which lines k and m intersect satisfies30°<|{tilde over (β)}_(k)−{tilde over (β)}_(m)|<150°.

Similarly, the transformed point ({tilde over (x)}_(km),{tilde over(y)}_(km)) corresponding to the intersection point (x_(lm),y_(km)) isgiven by: $\begin{matrix}{\begin{pmatrix}{\overset{\sim}{x}}_{km} \\{\overset{\sim}{y}}_{km}\end{pmatrix} = {\frac{1}{\sin\left( {{\overset{\sim}{\beta}}_{k} - {\overset{\sim}{\beta}}_{m}} \right)}\begin{pmatrix}{\sin\quad{\overset{\sim}{\beta}}_{k}} & {{- \sin}\quad{\overset{\sim}{\beta}}_{m}} \\{\cos\quad{\overset{\sim}{\beta}}_{k}} & {\cos\quad{\overset{\sim}{\beta}}_{m}}\end{pmatrix}{\begin{pmatrix}{\overset{\sim}{r}}_{m} \\{\overset{\sim}{r}}_{k}\end{pmatrix}.}}} & (20)\end{matrix}$

By substituting the parameter pairs {r_(n),β_(n)} and {{tilde over(r)}_(n),{tilde over (β)}_(n)} into Equations (19) and (20)respectively, and then solving Equation (2) using the correspondingintersection points (x_(km),y_(km)) and ({tilde over (x)}_(km),{tildeover (y)}_(km)), the affine distortion parameters {α₁₁, α₁₂, α₂₁, α₂₂,x₀,y₀} may be obtained.

The preferred method of affine parameter estimation uses least squaresfitting (LSF) between the intersection points (x_(km),y_(km)) of thelines and the intersection points ({tilde over (x)}_(km),{tilde over(y)}_(km)) after affine distortion. This LSF is greatly eased becausethe correspondence between the intersection points (x_(km),y_(km)) and({tilde over (x)}_(km),{tilde over (y)}_(km)) is known. Without theunique identification of the correspondence between intersection points(x_(km),y_(km)) and ({tilde over (x)}_(km),{tilde over (y)}_(km)) it isnecessary to evaluate the LSF for all possible permutations, in thiscase 6!=720.

Let the detected and (uniquely) ordered intersection points be denotedby ({circumflex over (x)}_(km),ŷ_(km)). As before the original(undistorted) intersection points are (x_(km),y_(km)), while thedistorted intersection points are ({tilde over (x)}_(km),{tilde over(y)}_(km)). An error energy is defined as the Euclidean norm measure E:$\begin{matrix}{E = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad\left\{ {\left( {{\hat{x}}_{km} - {\overset{\sim}{x}}_{km}} \right)^{2} + \left( {{\hat{y}}_{km} - {\overset{\sim}{y}}_{km}} \right)^{2}} \right\}}}} & (21)\end{matrix}$

Error energy minimization with respect to a transformation parameter pgives six equations: $\begin{matrix}{{\frac{\partial E}{\partial p} = {{2{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad\left\{ {{\left( {{\hat{x}}_{km} - {\overset{\sim}{x}}_{km}} \right)\frac{\partial{\overset{\sim}{x}}_{km}}{\partial p}} + {\left( {{\hat{y}}_{km} - {\overset{\sim}{y}}_{km}} \right)\frac{\partial{\overset{\sim}{y}}_{km}}{\partial p}}} \right\}}}} = 0}},\quad{{at}\quad{minima}},} & (22)\end{matrix}$with the transformation parameter p being one of the parameters α₁₁,α₁₂, α₂₁, α₂₂, x₀ and y₀ (Equation (2)) defining the transformation.

Fortunately the 6 equations represented in Equation (22) simplifiesgreatly because of the partial derivatives with respect to p=α_(ij):$\begin{matrix}\left. \begin{matrix}{{\frac{\partial{\overset{\sim}{x}}_{km}}{\partial a_{11}} = x_{km}},{\frac{\partial{\overset{\sim}{x}}_{km}}{\partial a_{12}} = 0},{\frac{\partial{\overset{\sim}{x}}_{km}}{\partial a_{21}} = y_{km}},{\frac{\partial{\overset{\sim}{x}}_{km}}{\partial a_{22}} = 0},{\frac{\partial{\overset{\sim}{x}}_{km}}{\partial x_{0}} = 1},{\frac{\partial{\overset{\sim}{x}}_{km}}{\partial y_{0}} = 0}} \\{{\frac{\partial{\overset{\sim}{y}}_{km}}{\partial a_{11}} = 0},{\frac{\partial{\overset{\sim}{y}}_{km}}{\partial a_{12}} = y_{km}},{\frac{\partial{\overset{\sim}{y}}_{km}}{\partial a_{21}} = 0},{\frac{\partial{\overset{\sim}{y}}_{km}}{\partial a_{22}} = x_{km}},{\frac{\partial{\overset{\sim}{y}}_{km}}{\partial x_{0}} = 0},{\frac{\partial{\overset{\sim}{y}}_{km}}{\partial y_{0}} = 1}}\end{matrix} \right\} & (23)\end{matrix}$

The six equations represented in Equation (22) are then essentiallyequivalent to matching the following summations: $\begin{matrix}\left\{ \begin{matrix}{{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{x}}_{km}x_{km}}}} = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\overset{\sim}{x}}_{km}x_{km}}}}} \\{{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{x}}_{km}y_{km}}}} = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\overset{\sim}{x}}_{km}y_{km}}}}} \\{{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{y}}_{km}x_{km}}}} = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\overset{\sim}{y}}_{km}x_{km}}}}} \\{{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{y}}_{km}y_{km}}}} = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\overset{\sim}{y}}_{km}y_{km}}}}} \\{{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\hat{x}}_{km}}} = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\overset{\sim}{x}}_{km}}}} \\{{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\hat{y}}_{km}}} = {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\overset{\sim}{y}}_{km}}}}\end{matrix} \right. & (24)\end{matrix}$

Filling in the full transform parameters from Equations (22) and (23)gives a pair of 3×3 matrix equations with the same symmetric matrixmultiplier M: $\begin{matrix}{\begin{pmatrix}{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{x}}_{km}x_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{x}}_{km}y_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\hat{x}}_{km}}}\end{pmatrix} = {{\begin{pmatrix}{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{x_{km}x_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{x_{km}y_{n}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad x_{km}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{y_{km}x_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{y_{km}y_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad y_{km}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad x_{km}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad y_{km}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad 1}}\end{pmatrix}\begin{pmatrix}a_{11} \\a_{12} \\y_{0}\end{pmatrix}} = {M\begin{pmatrix}a_{11} \\a_{12} \\y_{0}\end{pmatrix}}}} & (25) \\{\begin{pmatrix}{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{y}}_{km}x_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{y}}_{km}y_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\hat{y}}_{km}}}\end{pmatrix} = {M\begin{pmatrix}a_{21} \\a_{22} \\y_{0}\end{pmatrix}}} & (26) \\{M = {\begin{pmatrix}S_{xx} & S_{xy} & S_{x} \\S_{xy} & S_{yy} & S_{y} \\S_{x} & S_{y} & S\end{pmatrix} = \begin{pmatrix}{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{x_{km}x_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{x_{km}y_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad x_{km}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{y_{km}x_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{y_{km}y_{km}}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad y_{km}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad x_{km}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad y_{km}}} & {\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad 1}}\end{pmatrix}}} & (27)\end{matrix}$

The inverse of matrix M is another symmetric matrix M⁻¹, where$\begin{matrix}{{M^{- 1} = {\frac{1}{M}\begin{pmatrix}{{{- S_{y}}S_{y}} + {SS}_{yy}} & {{- {SS}_{xy}} + {S_{x}S_{y}}} & {{S_{xy}S_{y}} - {S_{x}S_{yy}}} \\{{- {SS}_{xy}} + {S_{x}S_{y}}} & {{{- S_{x}}S_{x}} + {SS}_{xx}} & {{S_{x}S_{xy}} - {S_{xx}S_{y}}} \\{{S_{xy}S_{y}} - {S_{x}S_{yy}}} & {{S_{x}S_{xy}} - {S_{xx}S_{y}}} & {{{- S_{xy}}S_{xy}} + {S_{xx}S_{yy}}}\end{pmatrix}}}{and}} & (28) \\{{M} = {{\det\quad M} = {{{- {SS}_{xy}}S_{xy}} + {2S_{x}S_{xy}S_{y}} - {S_{xx}S_{y}S_{y}} - {S_{x}S_{x}S_{yy}} + {{SS}_{xx}S_{yy}}}}} & (29)\end{matrix}$

Inverting Equations (25) and (26), the final solution for the leastsquares estimate of all six distortion parameters is explicitly:$\begin{matrix}{\begin{pmatrix}a_{11} \\\quad \\a_{12} \\\quad \\x_{0}\end{pmatrix} = {M^{- 1}\begin{pmatrix}{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{x}}_{km}x_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{x}}_{km}y_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\hat{x}}_{km}}}\end{pmatrix}}} & (30) \\{\begin{pmatrix}a_{21} \\a_{22} \\y_{0}\end{pmatrix} = {M^{- 1}\begin{pmatrix}{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{y}}_{km}x_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{{\hat{y}}_{km}y_{km}}}} \\{\sum\limits_{m = 1}^{k - 1}\quad{\sum\limits_{k = 1}^{N}\quad{\hat{y}}_{km}}}\end{pmatrix}}} & (31)\end{matrix}$

The minimum error energy E may be calculated using Equation (21), andmay be compared with an allowable limit. If the error energy E isgreater than the allowable limit, then it is assumed that an affinematch has not been found. It is also possible to calculate maximum pointdeviations by comparing intersection points (x_(km),y_(km)) and ({tildeover (x)}_(km),{tilde over (y)}_(km)), which is useful for establishinga reliability measure of the estimated distortion. Another usefulparameter, especially with regard to reliability estimation, is thenormalized energy; namely the error energy E divided by the centred meansquare point distribution G, defined as follows for any set of pointsn=1→M: $\begin{matrix}{\frac{E}{G} = \frac{{\sum\limits_{n = 1}^{M}\quad\left( {{\hat{x}}_{n} - {\overset{\sim}{x}}_{n}} \right)^{2}} + \left( {{\hat{y}}_{n} - {\overset{\sim}{y}}_{n}} \right)^{2}}{{\sum\limits_{n = 1}^{M}\quad\left( {{\hat{x}}_{n} - {\overset{\_}{\hat{x}}}_{n}} \right)^{2}} + \left( {{\hat{y}}_{n} - {\overset{\_}{\hat{y}}}_{n}} \right)^{2}}} & (32)\end{matrix}$where the centroid of the measured point distribution is defined as ({circumflex over (x)} _(n), ŷ _(n)).

Accordingly, FIG. 3 shows a schematic flow diagram of a method 200 ofdetecting parameters of an affine distortion from an image having atleast 4 non-parallel straight-lines therein. In one implementation thenon-parallel straight-lines are embedded into the image in a manner inwhich the straight-lines are invisible to the human eye. FIG. 4 shows aschematic flow diagram of a method 300 of embedding the straight-linesusing watermarks.

The methods 200 and 300 are preferably practiced using a general-purposecomputer system 100, such as that shown in FIG. 5 wherein the steps ofthe methods 200 and 300 are implemented as software executing within thecomputer system 100. In particular, the steps of the method 200 and 300are effected by instructions in the software that are carried out by thecomputer system 100. The software may be stored in a computer readablemedium, including the storage devices described below, for example. Thesoftware is loaded into the computer from the computer readable medium,and then executed by the computer system 100. A computer readable mediumhaving such software or computer program recorded on it is a computerprogram product.

The computer system 100 is formed by a computer module 101, inputdevices such as a keyboard 102, mouse 103 and an imaging device 122,output devices including a printer 115 and a display device 114. Theimaging device 122 may be a scanner or digital camera used for obtaininga digital image. A Modulator-Demodulator (Modem) transceiver device 116is used by the computer module 101 for communicating to and from acommunications network 120, for example connectable via a telephone line121 or other functional medium.

The computer module 101 typically includes at least one processor unit105, and a memory unit 106, for example formed from semiconductor randomaccess memory (RAM) and read only memory (ROM). The module 101 alsoincludes an number of input/output (I/O) interfaces including a videointerface 107 that couples to the video display 114, an I/O interface113 for the keyboard 102, mouse 103 and imaging device 122, and aninterface 108 for the modem 116 and printer 115. A storage device 109 isprovided and typically includes a hard disk drive 110 and a floppy diskdrive 111. A CD-ROM drive 112 is typically provided as a non-volatilesource of data. The components 105 to 113 of the computer module 101,typically communicate via an interconnected bus 104 and in a mannerwhich results in a conventional mode of operation of the computer system100 known to those in the relevant art.

Typically, the software is resident on the hard disk drive 110 and readand controlled in its execution by the processor 105. In some instances,the software may be supplied to the user encoded on a CD-ROM or floppydisk and read via the corresponding drive 112 or 111, or alternativelymay be read by the user from the network 120 via the modem device 116.Still further, the software can also be loaded into the computer system100 from other computer readable media.

The method of 200 of detecting parameters of an affine distortion froman image comprising at least 4 non-parallel straight-lines therein andthe 300 of embedding the straight-lines using watermarks mayalternatively be implemented in dedicated hardware such as one or moreintegrated circuits performing the functions or sub functions of thereofSuch dedicated hardware may include graphic processors, digital signalprocessors, or one or more microprocessors and associated memories.

Referring first to FIG. 4, the method 300 of embedding thestraight-lines, each straight-line n being defined by the parameter pair{r_(n),β_(n)}, using watermarks starts in step 310 where a patternassociated with each straight-line n is generated.

When choosing the parameter pair {r_(nl ,β) _(n)} of the lines to beembedded, one option is to choose 4 lines having angles β_(n) that areuniformly distributed over the full angular range, i.e. at 0°, 45°, 90°,135°. FIG. 6 shows the 4 lines arranged in such a fashion.

A number of arrangements of the four straight-lines give highlysymmetric results. The arrangement shown is less symmetric. It containssix intersection points (x_(km),y_(km)) that can be distinguished from areflected and distorted version of itself. Fewer intersections arepossible in degenerate arrangements where more than two lines meet at apoint, but are not considered here.

Also shown on FIG. 6 are the lengths ξ_(n1) and ξ_(n2) of line segmentsfor n=l to 4, defined with reference to the length p of segment 600, interms of a further parameter e. The length ratios R_(n), defined inEquation (15), of line segments as a function of the parameter e arecalculated as shown in Table 1, together with the line length l_(n)defined in Equation (18): TABLE 1 Line n Length Ratio R_(n) Total LengthL_(n) 1 $\frac{e}{\sqrt{2} - e}$ $\sqrt{2}\rho$ 2$\frac{e}{\sqrt{2} - {2e}}$ $\left( {\sqrt{2} - e} \right)\rho$ 3$\frac{e\sqrt{2}}{1 - {e\sqrt{2}}}$ ρ 4 $1 - {e\sqrt{2}}$$\left( {2 - {e\sqrt{2}}} \right)\rho$

FIG. 7 shows the length ratios R_(n) of line segments as a function ofthe parameter e. It can be seen from FIG. 7 that the particular valuee≅0.31 gives 4 quite different and distinct length ratios R_(n). Anotherinteresting and useful set of length ratios R_(n) occurs for e=0.5. Inthis case all four straight-lines are equidistant from a centralfiducial point; essentially allowing a seventh point to be defined,although it is a point that does not remain truly invariant to shear oranamorphic scaling because the equi-distances are potentially unequallytransformed by affine distortions.

In the preferred implementation (e=0.31) 4 straight-lines are embeddedhaving parameters as follows: $\begin{matrix}\left\{ \begin{matrix}{{\beta_{1} = {{- 67.5}{^\circ}}},{r_{1} = {{- p}/\left( {2 + \sqrt{2}} \right)}}} \\{{\beta_{2} = {{+ 22.5}{^\circ}}},{r_{2} = {{p\left( {1 - e} \right)}/\sqrt{2}}}} \\{{\beta_{3} = {{- 22.5}{^\circ}}},{r_{3} = {p/\left( {2 + \sqrt{2}} \right)}}} \\{{\beta_{4} = {{+ 67.5}{^\circ}}},{r_{4} = {{- p}/\left( {2 + \sqrt{2}} \right)}}}\end{matrix} \right. & (33)\end{matrix}$where p is set to the image width. Note that the negative values ofr_(n) are equivalent to their positive counterparts, provided thecorresponding angle β_(n) is rotated by 180°.

The preferred choice of orientation angles β_(n) avoids any coincidencebetween any one straight-line n and image boundaries by the largestamount.

When the straight-lines are embedded as watermarks, each watermark hasto have the property that it can be detected even after the image intowhich it is embedded has been affine distorted. According to thepreferred implementation each pattern has variation in one directiononly, that direction being perpendicular to the straight-line n. Thepattern also has an axis, which may be an axis of (odd or even) symmetryor a nominated axis, which coincides with the line. The patterns aretypically generated from a one-dimensional basis function applied at thedirection of variation and repeated parallel to the axis.

In a preferred implementation, the basis function is a complexhomogeneous function of the form:g(v)=|v| ^(p+ia) =|v| ^(p) exp(iα log[|v|](34)

where v is a one-dimensional coordinate, which is not necessarilypositive, while α and p are constants. The basis function g(v) ispreferably attenuated in areas where the basis function has a frequencyabove the Nyquist frequency of the pattern image. Equation (34) may beconsidered as an amplitude function, amplitude modulating a phasefunction, with the phase function having a logarithmic phase. When sucha complex homogeneous function g(v) is scaled, say by a factor α, thescaling only introduces a complex constant factor as follows:g(αv)=α^(p+ia) g(v) (35)

The advantage of the complex homogeneous function is that theauto-correlation of the complex homogeneous function is directlyproportional to the cross-correlation of the complex homogeneousfunction with a scaled version of the complex homogeneous function. This‘scale-invariant’ property allows a watermark to be detected in an imageeven after a scale transformation by correlating the image with thebasis function.

FIG. 8 illustrates an example pattern generated using a real part of thebasis function of Equation (34) as follows:g(x,y)=Re{g(x cos β_(n) +y sin β_(n) −r _(n))}(36)

Note that masking has been applied at the area adjoining the axis ofsymmetry to remove values with a frequency above the Nyquist frequencyof the pattern image. Also illustrated is the relationship of thepattern with the straight-line it embeds. It would be understood thatthe straight-line itself does not form part of the pattern, and wouldnot be added to the image in the steps that follows. In illustrating theexample pattern, pattern values have been mapped to values in the rangeof [0, 255], with a value of 0 being represented as the colour black anda value of 255 being represented as the colour white.

Referring again to FIG. 4, also in step 310 the separate patterns aresummed together to form a pattern image. The pattern image may be storedafter generation in the memory 106 or HDD 110.

In order to embed patterns that are imperceptible to a human observer,the pattern image is retrieved from the memory 106 and is perceptuallymasked in step 320 by the processor 105 in order to greatly reduce thelevels of the patterns corresponding to regions of the image having lowintensity variation, and reduce by a lesser amount the levels of thepatterns corresponding to regions of the image having high intensityvariation. An example measure of intensity variation is the localgradient magnitude of the luminance in the image. Other measures includesecond partial derivatives of the luminance; local estimates of the“energy” or frequency content, local variance, and more sophisticatedestimates of human visual system masking.

The perceptually masked pattern image, which may be called a watermark,is added to the image, in step 330. If the image is a colour image, thenthe watermark is preferably added to the luminance part of a colourimage. This allows the watermark to survive when the watermarked imageis converted from colour to a greyscale representation. Alternatively,the watermark may be added to one or more of the R, G, B, H, V, S, u, vetc channels of the colour image, or any combination thereof. Apart fromsimple algebraic addition, addition of the watermark to the image alsoincludes dithering and half-toning. The real and imaginary parts of acomplex basis function may be added independently to two or morechannels of the colour image.

The image with the embedded watermark may be stored on storage device109 (FIG. 5) in a digital image format, such as tiff, gif, jpeg, mpegetc. The watermarked image may also be printed on printer 115 (FIG. 5)to create a hardcopy of the watermarked image, or may be communicated toanother computer module 101 using the network 120.

With the straight-lines embedded into the image, and the image havingbeen affine distorted, the straight-lines and their parameter pair{{tilde over (r)}_(n),{tilde over (β)}_(n)} have to be detected. Withthe straight-lines embedded using watermarks in the manner describedwith reference to FIG. 4, method 200 of detecting the parameters of theaffine distortion from the image is now described with reference to FIG.3. Method 200 starts in step 202 where a digital copy of the distortedimage is obtained. Typically the image may already be in a digital imageformat and stored on storage device 109 (FIG. 5). In such cases theimage is converted to an array of pixel values. If the image is still ina hardcopy format, such as a photograph, then the hardcopy image isfirst converted into a digital format by scanning the hardcopy using theimaging device 122 (FIG. 5).

Step 205 follows where the processor 105 undoes the perceptual maskingby first forming a perceptual mask from the image, and then emphasisingthe image with the perceptual mask by dividing the values of the imageby the corresponding values of the perceptual mask. It is noted that anapproximate perceptual mask is adequate.

A projective transform is then applied to the resulting image in step210. The projective transform accumulates energy by summing values alongstraight lines in the image. The Radon (or equivalently Hough) transformis one such projective transform that may be used in step 210 and isdefined as: $\begin{matrix}{{R_{\theta}\left\{ {h\left( {x,y} \right)} \right\}} = {\int_{- \infty}^{+ \infty}{\int_{- \infty}^{+ \infty}{{h\left( {x,y} \right)}{\delta\left( {r - {x\quad\cos\quad\theta} - {y\quad\sin\quad\theta}} \right)}{\mathbb{d}x}\quad{\mathbb{d}y}}}}} & (37)\end{matrix}$

In order to derive a convenient implementation of the Radon transformfor a discrete dataset, a useful correspondence between the projectionof the image function h(x,y) and the slices of the function's Fouriertransform is used, that correspondence being known as the“projection-slice theorem”.

The projection-slice theorem states that the one-dimensional Fouriertransform of a projection of a two dimensional function is equal to aradial slice of the two-dimensional Fourier transform of that function.Note that $\begin{matrix}{{H\left( {u,v} \right)} = {\int_{- \infty}^{+ \infty}{\int_{- \infty}^{+ \infty}{{h\left( {x,y} \right)}{\exp\left\lbrack {{- 2}{{\pi\mathbb{i}}\left( {{ux} + {vy}} \right)}} \right\rbrack}\quad{\mathbb{d}x}\quad{\mathbb{d}y}}}}} & (38)\end{matrix}$

wherein H(u,v) is the 2-D Fourier transform of image h(x,y). In thequasi polar space the angles are in the range (−π/2,π/2], while distanceis in the range (−∞,∞). By defining quasi-polar coordinates (q,φ) in theFourier domain, the lo coordinate transform is u=q cos φ, v=q sin φ, andone form of the projection-slice theorem is obtained for the Fourierpolar angle corresponding to the Radon projection angle φ=θ as follows:$\begin{matrix}{{\int_{- \infty}^{+ \infty}{{R_{\theta}(r)}{\exp\left\lbrack {{- 2}{{\pi\mathbb{i}}({rq})}} \right\rbrack}\quad{\mathbb{d}r}}} = {H\left( {{q\quad\cos\quad\theta},{q\quad\sin\quad\theta}} \right)}} & (39)\end{matrix}$

Equation (39) is useful because it allows estimation of (the Fouriertransform of) a Radon projection as a radial slice of the 2-D-FFT of adiscrete image. This suggests that a discrete Radon transform may beevaluated by first performing a 2-D FFT followed by a Cartesian to polarremapping, using a suitable interpolation—such as bicubic, or chirp-z—toperform the resampling.

The image h(x,y) having embedded therein N patterns based onone-dimensional basis function g(v) having an orientation angle {tildeover (β)}_(n) and perpendicular displacement {tilde over (r)}_(n), andignoring the pixel values of the image itself, may be written as:$\begin{matrix}{{h\left( {x,y} \right)} = {\sum\limits_{n = 1}^{N}\quad{g\left( {{x\quad\cos{\overset{\sim}{\beta}}_{n}} + {y\quad\sin{\overset{\sim}{\beta}}_{n}} - {\overset{\sim}{r}}_{n}} \right)}}} & (40)\end{matrix}$

When the projection transform is applied to a pattern having a variationin only one direction with that direction being at an orientation angle{tilde over (β)}_(n), then the values of the projection aresignificantly higher when the angle θ is equal to the orientation angle{tilde over (β)}_(n) compared to all other values of angle θ. FIG. 9illustrates the projection of the undistorted example patternillustrated in FIG. 8, plotted in the quasi-polar space. It can be seenthat the values of the projection are significantly higher when theangle θ is equal to the angle β_(n). The projection also has symmetryabout the offset distance r_(n).

Hence, by applying the Radon transforms on the functions in Equation(40), it can be shown that the Radon transform of such a function isconstrained to a line where θ={tilde over (β)}_(n) as shown in thefollowing: $\begin{matrix}{{R_{\theta}\left\{ {h\left( {x,y} \right)} \right\}} = {\sum\limits_{n = 1}^{N}\quad{{g\left( {r - {\overset{\sim}{r}}_{n}} \right)}{\delta\left( {\theta - {\overset{\sim}{\beta}}_{n}} \right)}}}} & (41)\end{matrix}$

Having concentrated the image function h(x,y) having the embeddedpatterns g(v) onto N single lines in quasi-polar space by the use ofRadon transform, it is further possible to concentrate the energycontained in each of the lines into a single point (or into a smallregion near a point) by using 1-D quasi-radial correlations (incoordinate r) detection for all values of the polar angle θ.

Accordingly, step 220 follows where the processor 105 performs 1-Dcorrelations between the projection and the basis function g(v) in thequasi-radial coordinate r for all possible values of the polar angle θ.The term “correlation” also includes phase correlation and phasecorrelation scaled by energy. The resulting correlations have peaks atquasi-polar coordinates ({circumflex over (r)}_(n),{circumflex over(β)}_(n)).

In step 230 the processor 105 finds the absolute peaks of thecorrelation. The orientation angle {circumflex over (β)}_(n) andperpendicular displacement {circumflex over (r)}_(n) of each distortedembedded pattern is directly available from the quasi-polar coordinates({circumflex over (r)}_(n),{circumflex over (β)}_(n)) of the peaks.Also, the parameters pair {r_(n),β_(n)} of the embedded straight-linesare available.

The pre-processing stage of the detector finds a number of candidatelines, typically about 64, based on the highest correlation magnitudepeaks. From this number, all combinations of four lines are scrutinized.The number of possible combinations is 64!/(60!4!)=635376. If the anglesof the four lines do not all differ by at least 15°, then thatcombination is dismissed and does not pass on to the next stage.Assuming a combination of lines passes this stage, then the sixintersection points are calculated using Equation (19) and the four setsof line segment ratios {circumflex over (R)}_(n), evaluated. The ratios{circumflex over (R)}_(n) are then placed in order of increasing size. Amerit function for the closeness of fit to the expected line ratios(also placed in order of increasing size) is then evaluated as:$\begin{matrix}{\sum\limits_{n = 1}^{4}\left( {\frac{{\hat{R}}_{n}}{R_{n}} - 1} \right)^{2}} & (42)\end{matrix}$

All combinations with a merit function below a certain threshold(preferably 0.1) are labelled as “candidate combinations” for furtherprocessing.

Note that other merit functions could be used, for example the abovemerit function could be weighted to reflect the line strength (in termsof the line correlation magnitude) and thus reduce contributions fromweakly detected lines.

The following processing is then carried out fro each candidatecombination.

In step 240 the processor uniquely identifies the correspondence betweeneach straight-line n before and after affine distortion. In thepreferred implementation the length ratios {circumflex over (R)}_(n) ofthe lines after distortion are matched with the length ratios R_(n)before distortion and in a manner described with reference to Equation(15) are used to uniquely identify the correspondence between eachstraight-line before and after affine distortion.

Another technique that may be used to uniquely identify thecorrespondence between each straight-line before and after affinedistortion is to use basis functions having different parameters toembed each line. In such an implementation steps 220 and 230 arerepeated with each basis function.

Using Equation (20) and in step 250 the processor 105 next calculatesthe intersection points ({circumflex over (x)}_(km),ŷ_(km)). Finally, instep 260, the affine distortion parameters {α₁₁,α₁₂,α₂₁,α₂₂,x₀,y₀} areestimated using LSF described above with reference to Equations (21) to(32).

The final affine distortion parameters are chosen to be those producingthe lowest minimum error energy E (Equation (21)) over all candidatecombinations.

Once the affine distortion parameters {α₁₁, α₁₂, α₂₁, α₂₂, x₀, y₀} areestimated, the affine distortions may be inverted, sometimes calledrectification or registration.

Once the affine distortions are inverted, additional patterns, namedauxiliary patterns, that existed in the image prior to adding thepatterns in step 330 (FIG. 4) may be detected using conventionalmethods, such as the detection of pseudo random noise (PRN) sequencesvia the process of correlation. Such a process is also known asspread-spectrum watermarking, and is notorious for its extremesensitivity to geometric distortion. A few percent change in scale isusually enough to virtually destroy the correlation, and makes itimpossible to detect PRN in slightly distorted images.

However, the process of un-distorting and aligning the image involvesinterpolation and resampling, which are typically computationallyexpensive. Therefore, it is proposed to avoid rectifying the image byusing a complementarily distorted detection template when detecting theauxiliary patterns.

The major advantage of this approach is that the template may be definedin the Fourier domain. Affine distortions in the spatial domain lead tocorresponding affine distortions in the Fourier domain, along withlinear phase factors related to the translation distortion. The Fourieraffine correspondence is well documented in the literature. Essentiallythe template has the Fourier version of Equation (2) applied. Definingthe continuous Fourier transformation as follows, and noting thatexactly corresponding relations apply for discretely sampled images andthe discrete Fourier transform (DFT) and its more efficientimplementation, the fast Fourier transform (FFT):J(u,v)=∫∫j(x,y)exp(−2π[ux+vy])dxdy   (43)

The Fourier transform of a distorted template function iskJ(ũ,{tilde over (v)})=∫∫j({tilde over (x)},{tilde over(y)})exp(−2π[ux+vy])dxdy   (44)

The affine distortion of Equation (2) in the Fourier domain is given by$\begin{matrix}{\begin{pmatrix}\overset{\sim}{u} \\\overset{\sim}{v}\end{pmatrix} = {\frac{1}{{a_{11}a_{22}} - {a_{21}a_{12}}}\begin{pmatrix}a_{22} & {- a_{12}} \\{- a_{21}} & a_{11}\end{pmatrix}\begin{pmatrix}u \\v\end{pmatrix}}} & (45)\end{matrix}$wherein the factor κ contains a real normalization constant and a linearphase factor, but otherwise does not affect the distorted Fouriertemplate J(ũ,{tilde over (v)}), so that the distorted Fourier templateJ(ũ,{tilde over (v)}) is used directly in the Fourier domainimplementation of correlation detection. Hence, certain templates j(x,y)may be distorted using Equation (45) without the need for interpolationor resampling, if the Fourier template J(ũ,{tilde over (v)}) is definedby a deterministic, analytic functional descriptor. Only the Fouriercoordinates u and v need to be distorted prior to the template matching.

The auxiliary patterns may be used to encode some data or a pointer tothat data, for example using a URL or pointer to that URL. Also, usingthe method 200 of detecting the parameters of the affine distortion fromthe image, the so-called image metadata, which is data about, orreferring to some property of that image, is thus bound to the image inthat the metadata can be retrieved from the image even if the image isdistorted. The distortions that the metadata can resist include theprojective transforms above, but also include: Printing,Photocopying/copying, Scanning, Colour removal, gamma correction, gammachange, JPEG compression/general compression, format conversion (ie BMPto GIF), noise addition and removal, filtering, such as low-passfiltering, cropping, and almost any editing operations which maintainthe basic recognizability of the image.

Certain applications do not require for the straight-lines to beembedded imperceptibly. Also, the straight-lines may be embedded ontoany planar surface by engraving or etching. Such planar surfaces includesilicon wafers. The distorted straight-lines may then be detected bysome system which may be optical (such as a camera), electromagnetic, orproximity based. The detected straight-lines may be then utilized todetermine the surface's position and orientation.

The foregoing describes embodiments based upon the embedding of Nstraight lines. A revised version of the method 200 may also be appliedto images having straight lines therein associated with features in theimage. For such inherent lines, it is not possible to constrain, apriori, the four intersection ratios of each possible quartet of lines.It is however possible to compare the extant line ratios in a firstimage with those in a second image. For each quartet of matching ratios(within some predefined tolerance range) the corresponding affinetransform is calculated using steps 240 to 260 of the method 200. Steps240 to 260 are then repeated for matching ratio quartets. If thecorresponding affine transforms are consistent (i.e. the affineparameters show clustering), then it is probable that the two images arerelated to each other via an affine transformation. Hence it is possibleto compare two images and estimate whether the second image is relatedto the first image via an affine transformation.

Let a first image I₁(x,y) be related to a second image I₂(x,y) by anaffine transformation or spatial distortion (x,y)→({tilde over(x)},{tilde over (y)}), with the affine transformation being thatdefined in Equation (2). The relation between the first image I₁(x,y)and the second image I₂(x,y) may be written as follows:I ₂(x,y)=μI ₁({tilde over (x)},{tilde over (y)})+n(x,y)   (46)

wherein μ is an image intensity multiplying factor (or gain) and n(x,y)is function which takes account of the difference between the first andsecond images.

It is known that conventional methods of image matching (usingcorrelation for example) are not easy to implement for more generalaffine distortions, because a search over a 6 dimensional space (of the6 affine parameters) is required in general. Therefore, and inaccordance with the teachings herein, a more effective way to comparethe first image I₁(x,y) and the second image I₂(x,y) is by comparing theintersection ratios of naturally occurring line structures within theimages themselves. Many images contain some line structures or partialline structures, such as straight edges. Preferably such line structuresare enhanced, for example by using the modulus of the gradient of pixelintensities. Other operators, such as the Laplacian, may also be used.

FIG. 10 shows a schematic flow diagram of a method 400 of estimatingwhether the second image I₂(x,y) is an affine distorted version of thefirst image I₁(x,y). The method 400 is also preferably practiced usingthe general-purpose computer system 100 shown in FIG. 5, wherein thesteps of the method 400 are implemented as software executing within thecomputer system 100.

The method 400 starts in step 402 where a projective transform, such asthe Radon transform, is applied to each of the images I₁(x,y) andI₂(x,y), and in the manner described in relation to step 210 of themethod 200.

In practice it is found that enhancement after the Radon transformationis preferable. Accordingly, step 404 follows where gradient enhancementis performed giving the 2-D distribution function Ω_(j)(r,θ) for eachimage:Ω_(j)(r,θ)=|grad[R _(θ) {I _(j)(x,y)}]  (47)

In step 406 peaks are detected in each of the distribution functionsΩ_(j)(r,θ). The objective is to find corresponding quartets of linestructures. So by finding the 32 highest peaks within each of thedistribution functions Ω_(j)(r,θ), representing the 32 most significantline structures within each of the images I₁(x,y) and I₂(x,y), thenumber of possible combinations of 4 lines is 35960.

Step 408 then follows where the processor 105 calculates for eachcombination of 4 lines the intersection ratios, as is described indetail with reference to the method 200, from the (r,θ) values of the 4peaks. The 35960 combinations of ratio quartets are placed in orderedsequences in step 410. As before, if any ratio is greater than 1 it isinverted. The sequence is ordered in terms of increasing value. Inparticular, all combinations of ratio quartets are placed in a fourdimensional table for the first image I₁(x,y) and a separate table forthe second image I₂(x,y).

The ratio sequences for the first image I₁(x,y) and the second imageI₂(x,y) are next compared in step 412. In particular, the correspondenceof each ordered sequence in the first image I₁(x,y) table with those inthe second image I₂(x,y) table 2 is evaluated, based on Euclideandistance or the merit function shown in Equation (42). Onlycorrespondence within a predefined distance is allowed. The calculationof merit function is rather computationally intensive and is alsorelated to the square of the number of points (sequences) tested. A moreefficient approach is to have a coarse binning of the 4-D ratio table(for example bin widths of 0.2 will give 5⁴=625 bins) and just toevaluate merit function of sequences in corresponding or adjacent bins.

If the merit function is less than the predefined amount, then the linequartet is assumed to match and the consequent affine transformationparameters are computed in step 414 from the intersection points usingEquations (21) to (32). Step 414 is repeated for all matched sequencessatisfying the merit conditions and the 6 affine parameters aretabulated.

Step 416 follows where the likelihood of an affine relation between thefirst image I₁(x,y) and the second image I₂(x,y) is estimated. Forexample, if there is a significant clustering of affine parameterentries in the table, then an affine relation or match between the firstimage I₁(x,y) and the second image I₂(x,y) is likely. A morequantitative estimate of the likelihood of affine matching can bederived from the clustering statistics.

If the match between the first image I₁(x,y) and the second imageI₂(x,y) is considered not likely in step 416, then the method 400 endsin step 417 for example by indicating on a user interface that no matchbetween the images I₁(x,y) and image I₂(x,y) exists. However, if thematch between the first image I₁(x,y) and the second image I₂(x,y) isconsidered likely, then the first image I₁(x,y) is inverse affinedistorted in step 418 so that it matches the second image I₂(x,y).

Finally, in step 420, the quality of the match is determined bynormalised correlation. A value of 0.9 would indicate a good match.Other measures (e.g. visual) may give a better indication of how wellthe images match and how well the affine distortion parameters have beenestimated.

The foregoing describes only some embodiments of the present invention,and modifications and/or changes can be made thereto without departingfrom the scope and spirit of the invention, the embodiments beingillustrative and not restrictive.

In the context of this specification, the word “comprising” means“including principally but not necessarily solely” or “having” or“including”, and not “consisting only of”. Variations of the word“comprising”, such as “comprise” and “comprises” have correspondinglyvaried meanings.

1. A method of estimating an affine relation between a first image and asecond image, said first and second image each having at least 4non-parallel lines therein, said method comprising the steps of:identifying a first set of intersection points of said lines appearingin said first image; identifying a second set of intersection points ofsaid lines appearing in said second image; determining whether arelation between intersection points from said first and second sets ofintersection points exists, wherein if said relation exists then saidfirst and second images are affine related.
 2. A method as claimed inclaim 1 comprising a further step of estimating affine distortionparameters using said first and second sets of intersection points.
 3. Amethod as claimed in claim 2 comprising a further step of invertingaffine distortions, said affine distortions being defined by said affinedistortion parameters.
 4. A method as claimed in claim 3 comprising afurther step of determining a measure of said relation by normalisedcorrelation.
 5. A method as claimed in claim 1 wherein each of saidlines is embedded said first image and has a unique line segment ratiobetween intersection points with the other lines embedded in said firstimage, said determining step comprising the sub-steps of: determiningthe line segment ratio of each line in said second image; matching linesin said first and second images having corresponding line segmentratios; and identifying relation between said intersection points fromsaid first and second sets of intersection points.
 6. A method asclaimed in claim 1 or 5 wherein each of said lines are embedded in saidfirst image by embedding a pattern in said first image, each patternhaving been formed substantially from a one-dimensional basis function,said method comprising the initial steps of: calculating a protectivetransform of said second image; calculating a 1-D correlation betweenthe protective transform and said basis function for a selection angles;and finding peaks of said correlation, wherein the position of each ofsaid peaks provides spatial parameters of one of said embedded patterns.7. A method of detecting an auxiliary pattern in a second image, saidsecond image being affine distorted version of a first image having saidauxiliary pattern and at least 4 non-parallel lines therein, said linesforming a first set of intersection points at predetermined positions,said method comprising the steps of: identifying a second set ofintersection points of said lines appearing in said second image;identifying relation between intersection points from said first andsecond sets of intersection points; estimating said affine distortionparameters using said first and second sets of intersection points;applying said affine distortions to said auxiliary pattern to form adistorted auxiliary pattern, said affine distortions being defined bysaid affine distortion parameters; and detecting said distortedauxiliary pattern in said second image.
 8. A method as claim in claim 7wherein each of said lines is embedded in said first image and has aunique line segment ratio between intersection points with the otherlines embedded in said first image, said step of identifying relationbetween intersection points comprising the sub-steps of: determining theline segment ratio of each line in said second image; matching lines insaid first and second images having corresponding line segment ratios;and identifying relation between said intersection points from saidfirst and second sets of intersection points.
 9. A method as claimed inclaim 7 or 8 wherein each of said lines are embedded in said first imageby embedding a pattern in said first image, each pattern having beenformed substantially from a one-dimensional basis function, said methodcomprising the initial steps of: calculating a projective transform ofsaid second image; calculating a 1-D correlation between the projectivetransform and said basis function for a selection of angles; and findingpeaks of said correlation, wherein the position of each of said peaksprovides spatial parameters of one of said embedded patterns.
 10. Anapparatus for estimating an affine relation between a first image and asecond image, said first and second images each having at least 4non-parallel lines therein, said apparatus comprising: means foridentifying a first set of intersection points of said lines appearingin said first image; means for identifying a second set of intersectionpoints of said lines appearing in said second image; means fordetermining whether a relation between intersection points from saidfirst and second sets of intersection points exists, wherein if saidrelation exists then said first and second images are affine related.11. An apparatus as claimed in claim 10 further comprising means forestimating affine distortion parameters using said first and second setsof intersection points.
 12. An apparatus as claimed in claim 11 furthercomprising means for inverting affine distortions, said affinedistortions being defined by said affine distortion parameters.
 13. Anapparatus as claimed in claim 12 further comprising means fordetermining a measure of said relation by normalised correlation.
 14. Anapparatus as claimed in claim 10 wherein each of said lines is embeddedin said first image and has a unique line segment ratio betweenintersection points with the other lines embedded in said first image,said means for determining comprising: means for determining the linesegment ratio of each line in said second image; means for matchinglines in said first and second images having corresponding line segmentratios; and means for identifying relation between said intersectionpoints from said first and second sets of intersection points.
 15. Anapparatus for detecting an auxiliary pattern in a second image, saidsecond image being an affine distorted version of a first image havingsaid auxiliary pattern and at least 4 non-parallel lines therein, saidlines forming a first set of intersection points at predeterminedpositions, said apparatus comprising: means for identifying a second setof intersection points of said lines appearing in said second image;means for identifying relation between intersection points from saidfirst and second sets of intersection points; means for estimating saidaffine distortion parameters using said first and second sets ofintersection points; means for applying said affine distortions to saidauxiliary pattern to form a distorted auxiliary pattern, said affinedistortions being defined by said affine distortion parameters; andmeans for detecting said distorted auxiliary pattern in said secondimage.
 16. An apparatus as claimed in claim 15 wherein each of saidlines is embedded in said first image and has a unique line segmentration between intersection points with the other lines embedded in saidfirst image, said means for identifying relation between intersectionpoints comprising: means for determining the line segment ratio of eachline in said second image; matching lines in said first and secondimages having corresponding line segment ratios; and means foridentifying relation between said intersection points from said firstand second sets of intersection points.
 17. A program stored on acomputer readable medium for estimating an affine relation between afirst image and a second image, said first and second images each havingat least 4 non-parallel lines therein, said program comprising: code foridentifying a first set of intersection points of said lines appearingin said first image; code for identifying a second set of intersectionpoints of said lines appearing in said second image; code fordetermining whether a relation between intersection points from saidfirst and second sets of intersection points exists, wherein if saidrelation exists then said first and second images are affine related.18. A program as claimed in claim 17 further comprising code forestimating affine distortion parameters using said first and second setsof intersection points.
 19. A program as claimed in claim 18 furthercomprising code for inverting affine distortions, said affinedistortions being defined by said affine distortion parameters.
 20. Aprogram as claimed in claim 17 wherein each of said lines is embedded insaid first image and has a unique line segment ratio betweenintersection points with the other lines embedded in said first image,said code for determining comprising: code for determining the linesegment ratio of each line in said second image; code for mating linesin said first and second images having corresponding line segmentratios; and code for identifying relation between said intersectionpoints from said first and second set of intersection points.
 21. Aprogram stored on a computer readable medium for detecting an auxiliarypattern in a second image, said second image being an affine distortedversion of a first image having said auxiliary pattern and at least 4non-parallel lines therein, said lines forming a first set ofintersection points at predetermined positions, said program: code foridentifying a second set of intersection points of said lines appearingin said second image; code for identifying relation between intersectionpoints from said first and second set of intersection points; code forestimating said affine distortion parameters using said first and secondsets of intersection points; code for applying said affine distortionsto said auxiliary pattern to form a distorted auxiliary pattern, saidaffine distortions being defined by said affine distortion parameters;and code for detecting said distorted auxiliary pattern in said secondimage.
 22. A program as claimed in claim 21 wherein each of said linesis embedded in said first image and has a unique line segment ratiobetween intersection points with the other lines embedded in said firstimage, said code for identifying relation between intersection pointscomprising: code for determining the line segment ratio of each line insaid second image; code for matching lines in said first and secondimages having corresponding line segment ratios; and code foridentifying relation between said intersection points from said firstand second sets of intersection points. 23.-24. (canceled)